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WHAT IS CLAIMED IS : 

1 1 . A method for forming a consistency group of data, comprising: 

2 providing information on a consistency group relationship indicating a plurality of 

3 slave controllers and, for each indicated slave controller, a slave storage unit managed by 

4 the slave controller; 

5 transmitting a command to each slave controller in the consistency group 

6 relationship to cause each slave controller to transmit data in the slave storage unit to a 

7 remote storage in a manner that forms the consistency group; and 

8 determining whether all the slave controllers successfully transmitted the data in 

9 the slave storage units that is part of the consistency group to the remote storage. 

1 2. The method of claim 1 , wherein the remote storage is located at a remote 

1 site and wherein each slave storage unit is stored within a storage system attached to one 

2 slave controller. 

1 3. The method of claim 1, wherein the operations of providing information 



2 on the consistency group relationship, transmitting the commands to each slave 

3 controller, and determining whether all the slave controller successfully transmitted the 

4 data in the slave storage units to form the consistency group are performed by a master 

5 controller in data communication with the slave controllers. 



1 4. The method of claim 3, wherein the master controller also comprises one 

2 slave controller managing one slave storage unit including data to transmit to the remote 

3 storage, 

1 5. The method of claim 1, wherein each slave controller maintains a first data 

2 structure indicating updated data in the slave storage unit, wherein the slave controller 

3 transmits data in the slave storage unit indicated in the first data structure to the remote 

4 storage, further comprising: 
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5 transmitting a command to each slave controller to cause the slave controller to 

6 generate a second data structure to indicate any writes received to the slave storage unit 

7 during the transmittal of data in the slave storage unit indicated in the first data structure 

8 to the remote storage. - . 

1 6. The method of claim 1, further comprising: 

2 transmitting a command to each slave controller to cause each slave controller to 

3 initiate an operation to cause the data from the slave storage unit at the remote storage to 

4 be copied to a remote backup storage after determining that all the slave controllers have 

5 successfully transmitted the data in the slave storage xinits to the remote storage. 

1 7. The method of claim 6, further comprising: 

2 determining whether the data from the slave storage units at the remote storage 

3 were successfully copied to the backup remote storage; and 

4 transmitting a command to each slave controller whose slave storage unit data 



5 was successfully copied to the remote backup storage to vmdo the copying of the slave 

6 storage unit data from the remote storage to the remote backup storage in response to 

7 determining that the data from the slave storage units at the remote storage were not 

8 successfully copied to the remote backup storage. 



1 8. The method of claim 6, wherein the copy operation from the remote 

2 storage to the remote backup storage comprises a virtual copy operation. 

1 9. The method of claim 1, wherein the remote storage comprises a plurality 

2 of remote storage systems, wherein each remote storage system is coupled to one or more 

3 remote storage controllers, wherein each slave controller transmits data to one or more 

4 remote storage controllers to store the slave storage unit data in the remote storage 

5 system coupled to the remote storage controller, and wherein each storage unit comprises 

6 a volume of data. 
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1 10. The method of claim 1, further comprising: 

2 copying data from local controllers to the slave controllers to store in the slave 

3 storage luiits, wherein the local controllers and slave controllers are at different 

4 geographi'cal'sites and the remote storage is at a remote geographical location with 

5 respect to the geographical sites including the local and slave controllers. 

1 1 1 . A method for forming a consistency group, comprising: 

2 receiving a command from a master controller to generate a first data structure to 

3 indicate updates to a slave storage unit to form a consistency group initiated by the 

4 master controller; 

5 generating the first data structure; 

6 transmitting complete to the master controller after generating the first data 

7 structure; 

8 copying updated data in the slave storage unit indicated in a second data structure 

9 to the remote storage, wherein the data is copied to form the consistency group; and 

10 transmitting complete to the master controller after successfully copying the data 

1 1 in the slave storage unit to the remote storage. 

1 12. The method of claim 11, further comprising: 

2 indicating in the second data structure that data in the slave storage unit is not 

3 updated in response to transmitting the corresponding data to the remote storage; 

4 indicating in the second data structure updated data in the slave storage unit that 

5 is indicated as updated in the first data structure in response to the second data structure 

6 not indicating any updated data to copy to the remote storage; and 

7 indicating updates to data in the slave storage unit in the second data structure 

8 that are received after completing the copying of the data to the remote storage. 

1 13. The method of claim 1 1 , wherein the command received from the master 

2 controller comprises a first command, further comprising: 
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3 receiving a second command from the master controller to cause the copying of 

4 the slave storage unit data in the remote storage to a remote backup storage after 

5 transmitting the complete indicating that the slave storage unit data was successfully 

6 copied to the remote storage; and ~ 

7 transmitting a third command to a remote controller managing the remote storage 

8 to cause the copying of the slave storage unit data in the remote storage to the remote 

9 backup storage in response to receiving the third command. 

1 14. The method of claim 1 1, further comprising: 

2 queuing updates to the slave storage unit received while generating the first data 

3 structure; 

4 applying the updates to the slave storage unit after generating the first data 

5 structure; and 

6 indicating the queued updates applied to the slave storage unit in the first data 

7 structure. 

1 15. A system in communication with a plurality of slave controllers, wherein 

2 each slave controller manages at least one slave storage unit, and wherein each slave 

3 storage controller is in communication with one remote storage, comprising: 

4 means for providing information on a consistency group relationship indicating a 

5 plurality of the slave controllers and, for each indicated slave controller, at least one 

6 slave storage unit managed by the slave controller; 

7 means for transmitting a command to each slave controller in the consistency 

8 group relationship to cause each slave controller to transmit data in the slave storage unit 

9 to one remote storage in a manner that forms the consistency group; and 

10 means for determining whether all the slave controllers successfully transmitted 

1 1 the data in the slave storage units that is part of the consistency group to the remote 

12 storage. 
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1 16. The system of claim 15, wherein the means for providing information on 

2 the consistency group relationship, transmitting the commands to each slave controller, 

3 and determining whether all the slave controller successfully transmitted the data in the 

4 " slave storage units to form the consistency group are performed by a master controller in 

5 data communication with the slave controllers. 



1 17. The system of claim 16, wherein the master controller also comprises one 

2 slave controller managing one slave storage unit including data to transmit to the remote 

3 storage. 

1 18. The system of claim 15, wherein each slaye controller maintains a first 

2 data structure indicating updated data in the slave storage unit, wherein the slave 

3 controller transmits data in the slave storage unit indicated in the first data structure to the 

4 remote storage, further comprising: 

5 means for transmitting a command to each slave controller to cause the slave 

6 controller to generate a second data structure to indicate any writes received to the slave 

7 storage unit during the transmittal of data in the slave storage unit indicated in the first 

8 data structure to the remote storage. 

1 19. The system of claim 15, further comprising: 

2 means for transmitting a command to each slave controller to cause each slave 



3 controller to initiate an operation to cause the data from the slave storage unit at the 

4 remote storage to be copied to a remote backup storage after determining that all the 

5 slave controllers have successfully transmitted the data in the slave storage units to the 

6 remote storage. 

1 20. The systein of claim 19, further comprising: 

2 means for determining whether the data from the slave storage imits at the remote 

3 storage were successfully copied to the backup remote storage; and 
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4 means for transmitting a command to each slave controller whose slave storage 

5 unit data was successfully copied to the remote backup storage to undo the copying of the 

6 slave storage unit data from the remote storage to the remote backup storage in response 

7 to determining that the data from the slave storage-units at-the remote storage were not 

8 successfully copied to the remote backup storage. 

1 21. The system of claim 1 9, wherein the copy operation from the remote 

2 storage to the remote backup storage comprises a virtual copy operation. 

1 22. The method of claim 1, wherein the remote storage comprises a plurality 

2 of remote storage systems, wherein each remote storage system is coupled to one or more 

3 remote storage controllers, wherein each slave controller transmits data to one or more 

4 remote storage controllers to store the slave storage unit data in the remote storage 

5 system coupled to the remote storage controller, and wherein each storage unit comprises 

6 a volume of data. 



1 23. The system of claim 1 5, further comprising: 

2 means for copying data from local controllers to the slave controllers to store in 

3 the slave storage units, wherein the local controllers and slave controllers are at different 

4 geographical sites and the remote storage is at a remote geographical location with 

5 respect to the geographical sites including the local and slave controllers. 

1 24. A system in communication with a master controller, comprising: 

2 a slave storage xmit; 

3 means for receiving a command from the master controller to generate a first data 

4 structure to indicate updates to the slave storage unit to form a consistency group 

5 initiated by the master controller; 

6 . means for generating the first data structure; 
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7 means for transmitting complete to the master controller after generating the first 

8 data structure; 

9 a second data structure indicating updated data in the slave storage unit; 

10 means for copying updated data Jn the slave storage-unit indicated -in the second 

1 1 data structure to the remote storage, wherein the data is copied to form the consistency 

12 group; and 

13 transmitting complete to the master controller after successfully copying the data 

14 in the slave storage unit to the remote storage. 

1 25. The system of claim 24, further comprising: 

2 means for indicating in the second data structure that data in the slave storage unit 

3 is not updated in response to transmitting the corresponding data to the remote storage; 

4 means for indicating in the second data structure updated data in the slave storage 

5 unit that is indicated as updated in the first data structure in response to the second data 

6 structure not indicating any updated data to copy to the remote storage; and 

7 means for indicating updates to data in the slave storage unit in the second data 

8 structure that are received after completing the copying of the data to the remote storage. 

1 26. The system of claim 24, wherein the command received fi-om the master 

2 controller comprises a first command, and wherein the system is in commimication with 

3 a remote controller managing the remote storage, fiirther comprising: 

4 means for receiving a second command from the master controller to cause the 

5 copying of the slave storage unit data in the remote storage to a remote backup storage 

6 after transmitting the complete indicating that the slave storage unit data was successfiiUy 

7 copied to the remote storage; and 

8 means for transmitting a third command to the remote controller managing the 

9 remote storage to cause the copying of the slave storage unit data in the remote storage to 
10 the remote backup storage in response to receiving the third command. 
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1 27. The system of claim 24, further comprising: 

2 means for queuing updates to the slave storage unit received while generating the 

3 first data structure; 

4 ' means for applying the updates to-the slave storage unit after generating the first 

5 data structure; and 

6 means for indicating the queued updates applied to the slave storage imit in the 

7 first data structure. 

1 28. An article of manufacture for forming a consistency group of data, 

2 wherein a plurality of slave controllers manage slave storage imits, wherein the article of 

3 manufacture causes operations to be performed, the operations comprising: 

4 providing information on a consistency group relationship indicating a plurality of 

5 the slave controllers and, for each indicated slave controller, one slave storage unit 

6 managed by the slave controller; 

7 transmitting a command to each slave controller in the consistency group 

8 relationship to cause each slave controller to transmit data in the slave storage unit to a 

9 remote storage in a manner that forms the consistency group; and 

10 determining whether all the slave controllers successfully transmitted the data in 

1 1 the slave storage units that is part of the consistency group to the remote storage; 

1 29. The article of manufacture of claim 28, wherein the remote storage is 

1 located at a remote site and wherein each slave storage unit is stored within a storage 

2 system attached to one slave controller. 

1 30. The article of manufacture of claim 28, wherein the operations of 

2 providing information on the consistency group relationship, transmitting the commands 

3 to each slave controller, and determining whether all the slave controller successfully 

4 transmitted the data in the slave storage units to form the consistency group are 

5 performed by a master controller in data communication with the slave controllers. 
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1 31. The article of manufacture of claim 30, wherein the master controller also 

2 comprises one slave controller managing one slave storage unit including data to transmit 

3 to the remote storage. 

1 32. The article of manufacture of claim 28, wherein each slave controller 

2 maintains a first data structure indicating updated data in the slave storage unit, wherein 

3 the slave controller transmits data in the slave storage unit indicated in the first data 

4 structure to the remote storage, wherein the operations further comprise: 

5 transmitting a command to each slave controller to cause the slave controller to 

6 generate a second data structure to indicate any writes received to the slave storage unit 

7 during the transmittal of data in the slave storage unit indicated in the first data structure 

8 to the remote storage. 

1 33. The article of manufacture of claim 28, wherein the operations further 

2 comprise: 

3 transmitting a command to each slave controller to cause each slave controller to 

4 initiate an operation to cause the data fi-om the slave storage unit at the remote storage to 

5 be copied to a remote backup storage after determining that all the slave controllers have 

6 successfully transmitted the data in the slave storage units to the remote storage. 

1 34. The article of manufacture of claim 33, wherein the operations further 

2 comprise: 

3 determining whether the data from the slave storage units at the remote storage 

4 were successfully copied to the backup remote storage; and 

5 transmitting a command to each slave controller whose slave storage unit data 

6 was successfully copied to the remote backup storage to undo the copyinjg of the slave 

7 storage unit data from the remote storage to the remote backup storage in response to 

8 determining that the data from the slave storage units at the remote storage were not 

9 successfully copied to the remote backup storage. 
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1 35. The article of manufacture of claim 33, wherein the copy operation from 

2 the remote storage to the remote backup storage comprises a virtual copy operation. 

1 36. ' The article of niariufacture of claim 28; wherein t^^ 



2 comprises a plurality of remote storage systems, wherein each remote storage system is 

3 coupled to one or more remote storage controllers, wherein each slave controller | 

4 transmits data to one or more remote storage controllers to store the slave storage unit 

5 data in the remote storage system coupled to the remote storage controller, and wherein | 

6 each storage unit comprises a volume of data. ^ 

1 37. . The article of manufacture of claim 28, wherein the operations further 

2 comprise: 

3 copying data from local controllers to the slave controllers to store in the slave : 

■■-I 

4 storage units, wherein the local controllers and slave controllers are at different J 

1 

5 geographical sites and the remote storage is at a remote geographical location with 

6 respect to the geographical sites including the local and slave controllers. 

1 38. An article of manufacture for forming a consistency group of data, < 

2 wherein the article of manufacture receives communications from a master controller and 

3 causes operations to be performed, the operations comprising 

4 receiving a conraiand from the master controller to generate a first data structure 

5 to indicate updates to a slave storage unit to form a consistency group initiated by the 

6 master controller; . 

7 generating the first data structure; 

8 transmitting complete to the master controller after generating the first data J 

9 structure; 

10 copying updated data in the slave storage unit indicated in a second data structure 

1 1 to the remote storage, wherein the data is copied to form the consistency group; and 



,31-- ' Atty. Docket No. TUC920030108US1 

Firm No. 0022.0054 

12 transmitting complete to the master controller after successfully copying the data 

13 . in the slave storage unit to the remote storage. 

1 - 39. The article of manufacture of claim 38, wherein the operations further 

2 comprise: 

3 indicating in the second data structure that data in the slave storage unit is not 

4 updated in response to transmitting the corresponding data to the remote storage; 

5 indicating in the second data structure updated data in the slave storage unit that 

6 is indicated as updated in the first data structure in response to the second data structure 

7 not indicating any updated data to copy to the remote storage; and 

8 indicating updates to data in the slave storage unit in the second data structure 

9 that are received after completing the copying of the data to tiie. remote storage. 

1 40. The article of manufacture of claim 38, wherein the command received 

2 from the master controller comprises a first command, wherein the operations further 

3 comprise: 

4 receiving a second command from the master controller to cause the copying of 

5 the slave storage unit data in the remote storage to a remote backup storage after 

6 transmitting the complete indicating that the slave storage unit data was successfully 

7 copied to the remote storage; and 

8 transmitting a third command to a remote controller managing the remote storage 

9 to cause the copying of the slave storage imit data in the remote storage to the remote 
10 backup storage in response to receiving the third command. 

1 41. The article of manufacture of claim 38, wherein the operations further 

2 comprise: 

3 queuing updates to the slave storage unit received while generating the first data 

4 structure; 
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5 applying the updates to the slave storage xmit after generating the first data 

6 structure; and 

7 indicating the queued updates applied to the slave storage unit in the first data 

8 - structure; - - _ . 



